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Program Evaluation is sizing up a situation 



"before making a judgment. It is doing homework 
"before making a decision. It is sorting out the 
pros and cons, selecting what's good or bad, right 
or wrong before taking the next step. As a manage- 
ment process, it is a tool for more rational beha.vior. 
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INTRODUCTION 



There are a scarcity of' data pertaining to organizational pursuit of* 
evaluative research. This paper documents the YMCA experience; in Chicago, 
where attempts are now "being made to relate costs "co . results of* programs 
and services. It is assumed that cost accounting depends on management 
objectives and their ramifications. Objectives are useless unless they 
fulfill real needs which are amenable to measurement. 

Being a multi -service agency, the YMCA of Metropolitan Chicago has 
to test and validate management by alternatives — in this case, alternatives 
in evaluative research pertaining to program effectiveness in fulfilling 
a wide range of human needs • This implies Quality rather than Quantity 
or just- efficiency of expenditure alone. Evaluating quality is a difficult 
task for both practitioners and researchers . This paper is a first attempt 
to doc ume nt and to learn from actual experience how to evaluate program 
quality. It is presented for discussion, which we hope will result in 
practical alternatives for program evaluation. 

The YMCA of Metropolitan Chicago, like many private voluntary agencies, 
has long been compiling membership and program statistics as an integral 
part of everyday operations. In 1971s the Planning— Programming— Budgeting 
System (FPBS) was advocated by the local Community Fund for allocating 
funds. The YMCA quickly subjected it to vigorous testing, validation, 

and modification all an attempt to account for the cost of quality programs 

and services . 

There was a feeling (just a feeling) in the YMCA organization, that 
something was missing between statistics and PPBS . Both approaches attempted 
to account for program efficiency by such input data as dollar amounts , 
numbers and types or program activities, frequency and rate of participation. 
Neither has f ull y accounted for "program effectiveness," which has 
histroi cally perplexed all human services in or out of the YMCA. 
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Group #2 of the Program Evaluation Task Force for the YMCA of Chicago 
was formed in March of 1971 to research and develop alternate models that 
would incorporate the evaluation of program effectiveness . The five-member 
group charted a "mini-plan" for six months time (March through August), 
and the members spent the bulk of their time interviewing, consulting, 
and documenting existing practices in program evaluation which is summarised 
rn this report. In view of the current practice, it appears that the YMCA 
of Metropolitan Chicago predominantly uses one approach, j ,e . , project 
evaluation . 

Project evaluation is comparing results with one's own self-selected 



goals and objectives. The criteria for effectiveness are broadly biased 
toward those who are immediately involved and benefited by the program. 

The common rationale is "If they (the participants and funders) ©re 
happy, we're happy." This kind of evaluation is basically self-centered. 

At worst it is isolationist in policy, and at best it contributes to 

organizational, maintenance, perhaps for very legitimate reasons. 

It became clear in early 1971 that public and private funding policies 



were increasingly geared to eliminating waste in vested, and often redundant, 
interests. Federal agencies, along with United/Corarnmity Funds , had already 
begun the crusade against such, waste of -resources among public and private 
agencies . The emphasis now was program impact evaluation assessing overall 
institutional effectiveness among projects having similar objectives , techniques 
population, and environment. Despite the imperfection of evaluative techniques, 
allocation of resources (funds) was increasingly the outcome of analyses 
and rating schemes . Based on the proposition that social development and 
rehabilitation should result in the social good of all, not just of members, 
the alternate methods of program evaluation required vigorous definitions 
and measures of effectiveness effectiveness in fulfilling individual 
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and community needs rather than an agnecy's 



self-selected viewpoints , 



coals and objectives . 



BACKGROUND 

In February of 1971 , a 15 -member Program Valuation Task Force was 
appointed by John Boot, President of the YMCA of Metropolitan Chicago, to 
■undertake an emerging assignment: 

" Development of program and membership reporting, evaluation, 

and information for internal management and for Community Fund reportxng 
is a major objective of the TKCP of Metropolitan Chicago in 1971.” 

Bill Kuntz, Vice President for Corporate Resoruces and Development, was 
assigned the responsibility for coordinating the work of the ?csk Force. 

Bill Kuntz perceived the job of the Task Force in three groupings: 

1. "Work with Archie Beck (Coordinator of Membership and Program Reporting 
System) to design the best possible program and membership statistics 
record system to give us at least the body count in various program 
classifications on a daily basis." 

2. "Develop a hard set of criteria for evaluation and assessment of 

operating units’ effectiveness. This could he done hy assessing the 
adequacies and inadequacies of recent evaluations of South Shore , Off- 
the-Street, Duncan, Isham, Wabash, Youth Action and assessment teams’ 
products of lU Center programs conducted xn May and June of 1969* 

3. ’’Work with Wiley Moore (staff member of the Chxcago Communxty Fund) 

and the Community Fund PPBS (Planning-Programming-Budgeting System) 
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model to develop an information model and cost accounting system 
adapted to the YMCA." 

Further, Bill Kuntz recommended that the Task Force be administrative- 
ly cleared to put in up to 1/2 day per week (recommending Monday mornings, 
from 9:C9 a.m. to noon) as a project team for the next six months (average 

of five hours available work time, including meetings, homework, research, 
etc . ) 

On March 8, 1971, the fifteen member Task Force met for the first time 
with Bill Kuntz and John Root. All agreed with the goals and objectives 
of their tasks, and voluntarily joined with pre-selected staff aides Archie 
Beck, Bob Soong, and Vi Henry, in tree 'separate groups, as follows: 

Group #1: Program and Membership Data Sy stem 

Archie Beck, Robert Freeman, Robert Freitag, Ruth Scheele, 
Matthew Ottaviano, and Burt Wiese. 

Group #2 : Program Evaluation System 

Bob Soong, Sam Richardson, Ed LaShea, Bemie Chaudior, and 
Phil Pokomy. 

Group #3: PPBS System for the YMCA 

Vi Henry , Frank Arendt , Dave Hub at a, and A1 Kasufkin. 
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Group #2 of -the Program Evaluation Task Force accepted their charge 



in a simple statement of objective : "To research and develop a tested 

program evaluation system within six months (March through August, 19T1) 
Having discussed and defined the problems, needs, conditions, and more 
specific objectives, a "mini -plan” was outlined on March 10 to direct 

the course of Gr. -- #2 action. (See Appendix A.) 

Group #2 decided to interview a cross-section of YMCA personnel- 
from field workers involved in program operations to top management 
involved in overall policy-making. The plan was to interview twenty 
people, and they, in turn, would be instructed to interview another 
four each, thus totalling a study population of one hundred. (See 
Appendix B.) Inquires were to be made about the organisation under study, 
about the relationships and functions of tasks performed by the interviewee 
and about any administrative tools for documenting the relative merits 



of such performance. To facilitate the process, an interview guide was 
px-epared. (See Appendix C.) 

On April 19, Group #2 members, along with the entire Tank Force, 
reviewed their work with President John Boot and Vice President Bill Hunts. 



All endorsed the plan of action and verbally pledged their support. John 
Root was pleased with it because of "the training i„volvement"-from 
field operations to top management. Acknowledging such moral support. 
Group K promised to deliver a report of existing conditions, as well as 
a proposal for alternate system requirements by the end of June. 
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- serious problems end set-bachs arose 
* spibe Of the moral supp ^ ^ ^ of a database, 

tn the ?rOCeSS ° f data "or the months of March through June. 

, ^v t he "xnini-pl 911 * or ™ 

as proposed by tne 

.. for these set-backs. 

There were several cause 

th e massive coordination of 100 

1. Thu data-colleetion plan req re4v dred time and 

interviews and documentation in ~ donating 

n cn ‘ vpIv for contactxngs 

effort almost exc supplying materials , instructing. 

information, interviewing, work, *» 

, . „ clearing contacts ana related 

time -scheduling, on ^ T ash Force 

uas able to perform all thes ^ ^ ^ h egin - 

time Job. Of the Group »Z volunte , CO nflict with their 

ning of data collection. - ^ W red, or 

, TTn-rlc for irtixcn xiiey 

regular assign s- ^ people to handle 

later ^signed, to perform. That 

the colossal amount of worK that had to be done. 

, 2 depended solely on good will and the voluntary 

The work of Group strictly accountable 

+ 100 people . Bobody was held strict y 

time and effor ^ Has the format and organisation of 

to a prescribed time • n o contracts for 

H orh well enough structured in advance . ^ . 

* .„ le rewards or penalties associated wi 
work , no tanglbl ao t lve ly cooperated. 

. .. _ a i fev responded quick y 

0nl y the exceptional rttff meters of the YMCA, 

By the end of June, twenty out ^ ^ 

+ , onS to top management, had x 

from field operations t for program evaluation. 

viewing tasks , with ample documentation for progr 
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viewing 



3. Among the professional ranks of the YMCA, there are two types of staff: 
the Accessibles and the Untouchables . The Accessibles are those who 
make themselves available for contact. They give of their time, 
leave messages, really talk to people and make all attempts to take 
care of immediate business. The Untouchables are unavailable, don t 
have any time, and can’t take care of business now. Group #2 could 
only count on the good will of a few Accessibles for interview and 
documentation, and therefore, failed to saturate the sample selection 
of 100 within the proposed time-frame. 

U. The top m an agement (the vice presidents) seemed unable to differentiate 
amon^, the distinct functions of Groups #1, #2, and #3. Each top 
officer held a different perspective of program evaluation, and in 
spite of nodding agreements with John Root's directive, pursued 
their own interest, directed their staff time and effort out of, 
rather than into, the work of Group #2. In this respect, the 
administrative reassignment of three members from Group #2 seriously 
impaired the intended team work. Could this be due to conflicts of 
interest between the vice presidents? Or were they conflicts between 
Group #2 members? The Task Force? Some of them? All of them? 

5. On July 16, members of Group #2 met with staff aides of Groups #1 and 
#3, together with Bill Kuntz, to review this document. Suggestions 
were made , to expand and clarify Alternatives with examples of YMCA 
applications; to make forthright value judgment on the suggested 
alternatives; to show some criteria and their definitions for example. 
Group #2 then went on to solicit comments and recommendations from 
members of the total Program Evaluation Task Force until August 2, 19T1- 

0 



No further suggestions were made 'by that date or since then. 



This document is not to he mistaken as a research report, since 
not enough data were collected. Resulting from "both manpower 
and logistical problems of the Task Force, we have turned to ex- 
ploring alternatives , concluding only with a sharper focus on 
future attempts in program evaluation . 
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FINDINGS 



From the meager sampling of twenty YMCA personnel. Group #2 was 
able to come up with a few findings. Of those twenty persons interviewed, 
two were vice presidents, two were district directors, seven were 
executives of various types , and nine were professionals in various fields 



What are the findings? 

There are two major types of programs under the auspices of the 
YMCA: U.S. Government sponsored programs, and the YMCA's regular-, on- 

going programs . As for the Government funded projects, aside from head 
count and cost accounting, there is not any "hard set of criteria for 
evaluation and assessment of operating 'units' effectiveness." Moreover, 
the criteria for funding or support vary from one program to another. 

For example, SET-GO, a "Talent Search" project of the YMCA, is viewed hy 
the Government as "impactful" for serving over 8 6 % of 1 ,800 students from 
families with annual incomes "below $4,000. From an economic viewpoint, 
it is also efficient— $99 s 9 71 from HEW", matched hy $34,481 for 1,800 
students . This amounts to a cost to the Government of a little over 
$55 per student, for what is regarded as a unique opportunity "to serve 
clients who cannot gain admission elsewhere." Broadly speaking, a project 
such as SET-GO qualifies for "inpact," "efficiency," and "uniqueness." 

It projects an altruitic image of a Christian socio-educational institution, 
and it is good "business on the part of government to fund and support 
such a redeeming mission. 



There do exist some criteria, to ensure program effectiveness . 
These criteria are secondary in most Government funded projects only 
because they are not closely scrutinized or tested hy the grantor — hut 
are required on paper. These are criteria concerning methods and 
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techniques used in the projects; criteria dealing with the proportion of 
clients from Model Cities; criteria concerned with the extent of client 
participation and indigenous responsibility; and criteria delineating 
minutiae about the project director (his background, work experience, 
special personal qualifications pertaining to the target population, 
salary as related to his position, his ethno-racial background, and so on) 
Government funded projects of the YMCA are, by and large, assessed 
or evaluated on the basis of broad and superficial socio-economic 
criteria which are not amenable to measurement or replicability. The 
real danger of such vague and unreliable assessment is that while it 
encourages the Government to hold an image or illusion of progress 
(as with group advancement from one academic level to the next, from 
training to job, from welfare to employment), there is no substance to 
prove personal fulfillment or sustained social benefits. The public 
reaction to pupil achievement m the public schools focuses on such 
irony — that students are being promoted for every year that they fall 
further behind. 

As for program evaluations of the regular YMCA activities, there 
are even broader and more subjective criteria. Other than head count 
and cost accounting, the criteria for effectiveness are, for example, 
"better understanding," "goal-setting," "frank attitude," "new ideas," 
and "decentralized programs." Program assessments range from pages of 
staff narrative to volumes of open-ended participant reactions to or 
judgments about .their programs. (See Appendix E.) 
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.As a member of* Group # 2 . put it: 



"Throughout the YMCA of* Metro Chicago and some of* its 'satellite' 
agencies , the preponderance of* information recorded is of an accounting 
and nose-count nature for the pxrrpose of controlling daily operations , 
hut most importantly to add legitimacy to program budgeting. 

"There is no information system in operation that deals with the 
quality of YMCA porgrams . However, the accounting might well be integrated 
with some interpretation of evaluative data. 

"Quality of programming is likely to be discussed in 'Rap and Yap’ 
sessions, as one observer so aptly describes them. The qualitative 
comments and judgements are not recorded to facilitate their retrieval 
as required by an information system. The inherent problem with rap and 
yap sessions is that they represent points by people — people with 
different sets of standards concerning what they will call success or 
failure. Assessing the quality of programming might well be similar to 
taking a Rorschach test in the sense that different people see different 
things in the same picture. 

"A credible, reliable evaluation of quality of programming could be 
a wedding of accounting and nose-counting data. These kinds of information 
are already being collected. A new method of assembling and presenting 
it in the form of reports may be sufficient. This type of qualitative 
system would minimize the on-going retrieval effort. If additional input 
is needed to pinpoint the in— depth quality of a particular kind of 
programming, then a supplementary, custon-made retrieval method could be 
used — interviewing and/or a sort of newspaper reporting effort, for 
example. In other words, computers could supply the number for reports 
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and the text could come from the supplementary efforts if they are needed 

"After working on the Task Force for more than three months, the 
indications are that top management's driving concern is to know more 
about the YMCA’s product so they can better sell it to the financial 
hackers. The alternate models referred here can supply them with 
n umb er and words, the rest is up to their abilities to sell. 



"Could YMCA possibly mean Yes 

Maybe 
*£ertainly 
Approximately? " 

Whatever the YMCA possibly means, one interviewer reported, "It was 
great. It gave me a chance to see and understand what others are doing 
in the Y. Everybody should go through this experience. 
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ALTEKKATTVES 



Private /voluntary agencies such as the YMCA have not demonstrated 
that program evaluation is useful, largely because it is considered to 
he a luxury item in the area of program planning and implementation. 

Also, since each program/project is regarded as standing on its own merit, 
comparability of* data is not deemed necessary. 

The Federal Government, on the other hand, may be th e biggest 
spender when it comes to program evaluation ( e.g. , $13 million per year- 
on Title I ESEA evaluation). However, it has little to show for its 

i 

expenditure. The Federal Government's experience has resulted in thousands 
of non-comparab 1 <= project findings with non-comparable methods of data 



In order to make program evaluation useful, here are some current 
alternatives that could apply to voluntary service agencies such as the 
YMCA. 

1. Project Evaluation 

The Y, like many service institutions and government funded projects, 
is prone to conduct program evaluation, when and if 'necessary , by its 
own objectives and criteria. Project evaluation serves the limi ted 
purposes of fund-raising and promotion of programs and services . It 
should be noted that public and private funders are becoming increasingly 
skeptical about such evaluation. 

Project evaluation is carried out on a small, local basis. Each 
program within each Y Center is evaluated by its individual characteristics 
The program within any one Center are barely comparable to each other. 
Comparison of programs between Centers is difficult except in terms of 
gross characteristics of inputs and nose-counting variables.. Evaluation 



collection ana interpretation. The outcome is similar to the YMCA 




experience. 



is irregular and uncoordinated, conditioned by recreational season and 
fiscal timing. 

If the purpose of a swimming program at Center A is to graduate 
twenty swimmers with life-saving certificates within six months time, 
and the purpose of a swimming program at Center B is to increase the 
water-confidence and swimming pleasure of thirty participants in th ree 
■ months time , you can add up the programs and participants : two sw immi ng 

instruction programs with fifty participants. Center C might not provide 
instruction, but simply open their pool for swimming two nights a week 
for boys of specified ages. You now have three swimming programs , and 
you can rate the success of the three in terms of their respective goals. 
However, you have little rationale for concluding that the Centers should 
repeat their programs. You have no rationale for recommending replication 
of another Center’s methods. You have no basis for fund-raising to 
expand the swimming programs on the contention that the Centers are 
meeting the needs of their communities — in short for any planning purposes 
Furthermore, you have no understanding of the reasons for success or 
failure of the programs, even in terms of their own goals. 

The personnel responsible for conducting the programs will be held 
accountable. In order that their programs be considered successful, they 
will state program goals vague^r or in an ex post facto fashion, finding 
reasons for success in their personal efforts and ingenuity, and reasons 
for failure Jn external conditions or circumstances beyond their control- 
racial conflict, poor weather, competition with public facilities, 
insufficient funds, or flu epidemics. Staff vulnerability is maximized 
by such evaluation and, as might be expected, their response is to prefer 

isolation and to resist judgment of their programs, which amounts to 
judgment of them. 
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2 . Comparative Evaluation 



Often called "Relative Effectiveness Evaluation," this approach 
begins with the broad questions of ,t Which progr ams are working best? 

Under what conditions? Which ones are working poorly or not at all? Why? 
Which types are producing the biggest payoff?" Strategy and Rating are 
crucial. 

The multi-million-dollar program evaluation of Title I (ESEA) pro- 
jects, for example, is becoming more useful by the institution of comparative 
evaluation techniques. It proposes to group local programs /projects into 
categories of similar conditions, problems, objectives, and modes of 
operation. The organization of these projects and their output data will, 
therefore, enable one to see which programs /projects are in the top or 
bottom 25 # within their respective category. 

A comparative evaluation of stategy and program rating is intended 
to de— emphasize unrelated evaluation or unrelated progr ams or projects , 
since such evaluation does not add up to a recognizable overall institutional 
effectiveness on social systems, i .e . , health, education, economics, and 
so on. Given some studies in depth, all programs could be classified and 
compared in this way. 

3. Experimental Evaluation 

Experimental evaluations are conducted primarily by the government 
in such pilot projects as income maintenance, performance contracting, 
and voucher systems of education. Experimental evaluation is useful for- 
introducing ma j or or expensive programs . It may serve as a forerunner, 
or it may run simultaneously with a large operating program, testing, 
validating, and. evaluating different methods and techniques, to provide 
alternate strategies , and to compare their relative merits . The new man- 
■^ower , youth opportunity, and community development projects of the 



YMCA are ideal for such evaluation. 

There is growing support for such an experimental hut definitive 
approach to program evaluation. 

4 . Program Monitoring 

This kind of program evaluation emphasizes rapid-feedback and 
monitoring of existing programs . It could he used as a management strategy 
to differentiate superior performance from inferior, and to direct, divert, 
or terminate activities accordingly. Its major features are l) an on-site 
classification of programs and projects, 2) a system of pre-site visit 
data collection, 3) an on-site system of data collection pertaining to 
quantity and quality of work toward program objectives, and 4) a format 
of rapid-feedhack to the operating staff and management. 

This approach is useful and helpful to program managers. Currently, 
consumer feedback is emphasized. Techniques such as opinion surveys , 
telephone interviews, group-screening, and interviews are being examined. 
All YMCA programs can profit from this type of program evaluation. 

5. Impact Evaluation 

Despite current attempts to apply to the social or human services 
such concepts as cost accounting, cost effectiveness, planning-programming- 
hudgeting, systems analysis, electronic or automated data processing 
system, the usual criteria for program impact have not all been convincing. 
It is not convincing, for example, to state that the longer a participant 

I 

stays in a program, the better are his chances of personal or social 
development; that the higher-frequency joiners of program activities are 
more likely to be better citizens; that the mere salaries, materials, 
and supplies that are invested, the greater are the chances for program 



success . 



Long "before the professionals popularized the jargon such as PPBS, 

MBO, and "cost-benefit analysis," much was said about "getting the most- 
out of every dollar." It was assumed, and still is, that money was 
limited, and need for services was unlimited. Based upon this assumption, 
there never was enough money for the necessary service. With a little 
professional expertise and political endorsement, this same assumption 
"became the Doctrine of Endless Unmet Needs , which always justified in- 
creasing input of resources , "but never quite accounted for output results . 

The public now demands a radical departure from the Unmet Needs 
Doctrine. Assuming for the moment that needs are limited, service 
objectives are therefore limited and specific. Ways and means can then 
be planned and implemented to achieve these objectives most economically. 
While our Federal Government continues to debate whether another billion 
dollars ■are better spent in education, or on health care, or for defense, 
service agencies on a smaller scale must debate and delimit their priorities , 
and determine the cost of desirable results . 

It is admittedly difficult to place a dollar cost upon behavioral 
objectives. However, comparisons of program impact can be made, if the 
quantifiable objectives or resources are held constant. For instance: 



Example A (holding objective constant) 



Objective: To raise the reading ability of 6.0 level students to 8.0 

level in six months. 



Method #1 operates 
Method #2 operates 
Method #2 operates 
Method operates 

These alternate methods could be 



§ $80 per enrollee 
§ $60 per enrollee 
@ $75 per enrollee 
§ $55 per enrollee 

used variously by the Neighborhood 



Youth Corps, the Central YMCA High School, the YMCA College, or a reading 
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clinic of a YMCA Center. Provided that the program participants are com- 
parable, and the same results are achieved by all four methods, the less 
costly program. Method #k , should be favored. 

Example B (holding resources constant) 

The same objective applies as in Example A. Each alternate method is 
allocated the same budget for the same time span, but different results 
are produced. 



Method #1 


succeeds 


with 


1 


out 


of k 


enrollees 


Method #2 


succeeds 


with 


1 


out 


of 6 


enrollees 


Method #3 


succeeds 


with 


1 


out 


Of 5 


enrollees 


Method 


succeeds 


with 


1 


out 


of 8 


enrollees 



Method #1, in this case, would be preferred. 

In all, there axe five broadly known criteria that fit into the 
above alternatives to program evaluation. They are effort (or efficiency), 
effectiveness, impact, cost effectiveness , and process . 

Effort » 

This program evaluation criterion usually attributes to the amount 
of workload or activities, viz., the number of man-hours, camper-days, 
rate of client intake, types of programs, and amounts of equipment, 
supplies, etc. It is to determine how much is put into program activities 
without regard to their outcome. Evaluation in this respect connotes 
efficiency rather than effectiveness of program effort. 

Effectiveness 

Effectiveness has to do with the adequacy of a certain amount of 
program effort rather than the effort itself. For example a special 
program may achieve a high effort (rate of efficiency) in producing 
youngsters capable of passing the college entrance Scholastic Aptitute 
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Test (based on verbal and mathematical skills). Yet, the same youths may 
result in the lack of overall creativity and imagination, initiative, 
sensitivity, political and social leadership, adaptability and commitment 
to responsibility. These latter attributes of effectiveness are grossly 
overlooked by human service agencies, perhaps, due to the .lack of reliable 
tests and measurements; and more likely, due to the "you can't measure 
the quality of our work" attitude. 

Impact 

Assuming that the program or service objectives are achieved by 
effective performance , impact is concerned with whether the overall needs 
are adequately met by such performance objectives. Take for example the 
emergency food program to fight hunger. It is handled with efficiency, 
and the pronounced goal that "nobody in Chicago should starve" is effectively 
met. What then is the impact of such a program upon self-sufficiency to 
buy food in the first place? What is its impact on nutrition and health? 

Cost Effectiveness 

This criterion is concerned with program or service effectiveness 
solely in terms of costs. Cost effectiveness is usually represented by 
a ratio between the cost of effort and the resulting impact . 

Process 

Process as a criterion attempts to describe what happens to the 
program that produces its results. It may serve to predict future results 
and their effects . 
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The five alternatj/ves to evaluation, therefore, rely on different 
criteria for achieving their respective purposes: 



EVALUATION 

ALTERNATIVES 




EVALUATION CRITERIA 


, Effort 


Effectiveness 


Impact 


Cost 


Process 


Pro.iect 




X 








Comparative 




X 




X 


X 


Experimental 




X 






X 


Monitoring 


X 


' 








Impact 




. J 


X 


X 





Program executives of the YMCA would he concerned with the monitoring 
of on-going program efforts and effectiveness as an evaluation alternative, 
whereas experimental and demonstration project directors would be more 
interested in the comparable processes toward program effectiveness. Cor- 
porate policy makers are more likely to use comparative and impact eval- 
nations toward long-term corporate advantages. 
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CONCLUSIONS 



Until now, the YMCA (like all others) has "been satisfied with 
professional standards, codes of ethics, working instinct, long-term 
experience , numbers , program reports , principles and creeds , as sufficient 
guarantees of program effectiveness. Our criteria for program effectiveness 
has been over-simplified: "If they (clients and funders) are happy, we're 

happy." Program evaluation, if any, has been solely based upon input 
statistics, viz . , dollar amount, number of clients, calls, visits, 
instructions, sessions, camper days, and so forth. 

There has not been the need for a change in program evaluation until 
now — a time of economic slump. Input statistics and professional 
qualifications are no longer adequate to the task of measuring effectiveness of 
output. Under legislative and consumer pressures, the Federal Government 
is now undergoing comprehensive changes from input-oriented documentation 
to output evaluation— —apply j.ng scientific methods to learn the performance 
and outcome of program activities . 

Under similar pressures. Community Funds across the nation are 
responding to the need to evaluate results, compare outcomes, relative 
effectiveness of output, as well as strategy. Rallying around the b ann er 
of the United Way of America, the Community /United Funds of this country 
are tapping the experience of Federal agencies (particularly the Social 
Rehabilitation Service of HUW ) to improve the capabilities of the 
PI ann ing-Pr o grammi n g-Budge ting System (PPBS). 

The major inadequacy of PPBS is, again, the over-emphasis on input 
documentation of such criteria as program efforts, effects, and efficiency, 
rather than program effectiveness with evaluative measures. Program 
efforts have again required the documentation of number and types per- 
taining to program activities.. Program effectiveness is still measured 
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in terms of whether or not intended objectives (regardless of their 
appropriateness) have been achieved. Program efficiency tends to stress 
the cost of various program activities. In the current confusion, 
there is the tendency to consider cost in terms of program efforts and 

» hence producing data on cost— efficiency rather than effectiveness . 
The methodological breakthrough anticipated by evaluative researchers 
is to relate impact and strategy to cost analysis in order to measure 
program effectiveness. This may be regarded as a diagnotic approach, 
to determine the "why’s” and ’’how's” of program results. 

The phrase ’’you can't measure the quality of our work" is no 
longer a valid excuse for neglecting evaluation because in both the 
voluntary and public service fields, programs are being planned to 
permit quantifiable measurements of results. Under the all-encompassing 
term, "systems approach," new tools and techniques are currently being 



Urban Institute 

System Development Corporation 
Institute for Health Research 
University Research Corporation 
Model Cities Programs 
Office of Economic Opportunity 
Social Rehabilitation Service (II.E.W.) 



to name just a few. , 

In its continuning attempts to be up-to-date, the YMCA has, with- 
in the last several years, been examining itself. But the only results 
have been intuitive and sporadic changes. Meanwhile, across the nation, 
largescale evaluations have clearly indicated the waste inherent in 
piecemeal programming that adopts individual, specific project evaluation, 
which produces results that are not comparable. This applies to the 
YMCA of Metropolitan Chicago. Coordination is still needed, irrespective 
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CONSIDERATIONS 



1. Leadership 

YMCA management should require the internal use of objective evidence 
for program effectiveness. Also, management should perodically 
make known their viewpoint on problems concerning effectiveness of 
program operations. 

2. Resources 

While the Y currently has high fixed costs and assets, some process 
such as "zero budgeting” might be a helpful start. Human, as well 
as financial, resources should be considered. An important resource 
involved in evaluation is the computer capacity of the Y. 

3. Program Evaluation System 

Management should require the review and approval of program 
evaluation plans, overall organizational plans, and special evaluative 
studies . 

• 4 

4;. Operational Support 

Each operating unit should project evaluation results as part of the 
budget proposal. Definitions and measurements should be compared, 
reviewed, articulated, and approved for program operations. 

5. Priorities of Evaluation 

Priorities should be determined on the bases of feasibility and 
practicality as indicated among different methods of program evaluation. 

6. Evaluation Policy 

a) All attempts should be made to evaluate comparable programs 

28 
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and projects. Effective operations should he encouraged, whereas 
others should he diverted, regrouped, and redirected, 
h) Overall program evaluation to serve any purpose should he 

supported hy project rating and strategy evaluation. All attempts 
should he made to determine the effectiveness of methodological 
approaches despite overall success or failure. 

c) Corporate resources should he diverted to support programs 
which are amenable to impact evaluation , strategy evaluation. 
and project rating . 

d) Experimental and demonstration projects should he supported only 
if they serve to test relative effectiveness of different 
strategies (or methodological approaches), 

7 • Role of Evaluation 

To implement a corporate evaluation policy, the evaluator should 
develop hoth short-term and long-range criteria for program effective- 
ness, for impact evaluation, for strategy evaluation, and for project 
rating. Since program evaluation is intended to help rather than hinder 
on-going operations, the administrative judgement pertaining to strategies 
and directions should rest with the appropriate decision-maker (not 
evaluator ) who is responsible for program operations. 

8. Logistics 

An important element in determining logistics is the integration 
of currently unrelated systems, including management hy objectives, 
accounting, planning and budgeting, and fund-raising. 

9. Quality of Life 

Agency staff and benefactors are prone to say that the purpose of 
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Yet, 



their work or contribution is to improve the "quality of life." 
few have been able to offer measures of such quality in understandable 
terms . The Urban Institute has developed single indicators for four- 
teen broad aspects of the quality of life, as follows: 



Aspect 



Indicator 



Poverty 
Unemployment 
Racial Equality 
Mental Health 
Health 

Traffic Safety 
Air Pollution 
Income Level 
Housing 

Social Disintegration 
Community Concern 
Public Order 
Education 

Citizen Participation 



% Low Income Households 
% Unemployed 

Non-White/White Unemployment Rates 

Suicide Rate 

Infant Mortality Rate 

Traffic Death Rate 

Air Pollution Index 

Per Capita Income 

Cost of Housing 

Narcotics Addition Rate 

Per Capita United Fund Contributions 

Reported Robbery Rate 

Draft Rejection Rate 

Presidential Voting Rate 



Admittedly, this set of indices cannot express the complex quality 
of everyday life. It is, nevertheless, a conceptual breakthrough 
for further refinement in evaluation research. 

10 . Dimens ions 

In an organization like the Y, there are many kinds of program 
evaluation. They should, for the most part, be policy— determined 
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as to why , how , how long , and how often evaluation will be carried out. 

PPBS is essentially a planning approach. It should be applied in 
conjunction with, but not in place of evaluation. Nose— counting 
statistics and cost-benefit analysis may (should) contribute to both 
PPBS and Evaluation. 



Evaluation would appear to have the following dimensions: 

l) General Purpose of Evaluation 

a. Institutional maintenance ("Monitoring" or "Project Evaluation") 

b. Institutional change ("Impact”) 

c. Improvement of methods and techniques ("Experimental") 

d. Comparing results ("Comparative") 



II ) Specific Purpose 

a. Planning 

b . Budgeting 

c . Manpower development 

d. Fund-raising and publicity 

e. Governmental requirements 

f. Etc. 

Ill) Impact of Evaluation (Processes affected by findings) 

a. Institutional purpose 

b . Goals 

c. Objectives 

d. Program planning 

e. Program implementation 

i 
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IV) Scope 



a. Project or Center 

b . Center of Community 

c. Center as compared with Center 

d. YMCA District or other 
c. Metropolitan YMCA 

f. Metropolitan "Community" 

V ) Focus 

a. Effort (input) 

b . Result ( output ) 

c. Effect (implication) 

VI) Continuity 

a. Ongoing 

1) Continuous 

2) Regular Intervals 

b. Occassional, or as needed 
(for publicity, fund-raising) 

c . Intervals determined by external fore 
( e.g. , Government, Community Fund) .. 

VII) Frequency 

a. Monthly 

b. Quarterly, Seasonally 

c . Annually 

d. Intervals 2-5 years 

e. Irregular 

i 





VIII) Timing 



a. Coordinated with fiscal year 

"b. Coordinated with fiscal year with external requirements 
( e.g. , Government fiscal year) 

c. Not coordinated. On its own schedule. 

d. As needed. Irregular. 

IX) Definition of Criteria 

a. by Program or Project Personnel 

b. by Center personnel (Director e.g. ) 

c. by Participants 

d. by local boards of directors 

e. by Metro-level personnel 

f. by the Government 

g. Other sources 

X) Evaluation to fulfill: 

a. Personal staff needs 

b . Organizational needs 

c. Client or consumer needs 

t 

d. Other needs . 
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MINI-PLAN - FLOW CHART OF SIX MONTH 
RESEARCH AND DEVELOPMENT OF PROGRAM 
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APPENDIX B. 




Northwest Youth Outreach 
of the YMCA of Metropolitan Chicago 
Administrative Office 
4854 West Addison Street, Chicago, Illinois 60641 
(312) 777-7112-3 



TO: Group #2; Program Evaluation Task Force 

FROM: Bernie Chaudoir 

RE: Interviewing Logistics 



This x s an attempt to pull together some ideas as to how we 
will go about gathering data for our program evaluation sys 



As decided at the March 23, 1971 meeting, the tool to be used 
w . i . involve interviewing. It is to the process of inter- 
viewing that these suggestions are made. 



The process herein proposed involves three phases: 

1. Pre-test of the tool by the Group #2 "Ad Hoc" Com- 
raittee. 

2. Interviewing of twenty persons (four by each of the 

Group # 2 members) who will serve as interviewers in 
Phase 3* 



interviewing of eighty persons (four by each of the 
Phase 2 persons). 

U * K 100 interviews beyond the Group 
I* 11 ? ulX Persons in Phase 2 who will help 
22 on t^ time demanded of the Task Force members and 

yet will not involve major pieces of time from them. 

descriptions of the three phases with some 
levels t0 indlcate the amount of time involved at each of the 



Bernie Chaudoir 
April 2, 1971 
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Leyden Unit 

9660 Franklin Avenue 

Franklin Park, Illinois 60131 466-3224 



North Unit/Ravenswood YMCA 
1726 West Wilson Avenun 
Chicago, llltnoi* 60640 t>6 T <ft360 




Northwest Youth Outreach 

of the YMCA of Metropolitan Chicago 

Administrative Office 

4854 West Addison Street, Chicago, Illinois 60641 
(312) 777-7112-3 



PHASE I 



Phase I involves a testing of the tool by the CroupM Members, 
to S?tiating d ?hLf 2 ! he APril 12 meeting and prior 



I would not see using the results of this 
purpose other than evaluating the tool and 
members of Group #2 with it. 



pre-test for any 
familiarizing the 



i 
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Unit 

Vest Addison St. 

Milinoit 60641 777-7112 



Leyden Unit 

9660 Franklin Avenue 

Franklin Park, Illinois 60131 466-3224 



North Unil/Ravonswood YMCA 
1726 West Wilson Avonue 
Chics qo, Illinois «K40 5<*T 6360 




Northwest Youth Outreach 

of the YMCA of Metropolitan Chicago 

Administrative Office 

4854 West Addison Street, Chicago, Illinois 60641 
(312) 777-7112-3 



PHASE II 



Phase II is the crux of the entire process. It will demand 
the highest degree of care since it will or will not lay 
the base for a good Phase III. It will involve three stages: 

Stage 1. Orientation. 

Each member of the first phase will be responsible 
for four members of Phase II. The Orientation ses- 
sion will present the tool, explain the interviewing 
process, assign persons for each to interview, and 
set dates for their personal interviews by a member 
of Group #2. 

Time: 2 hours. 

/ 

Stage 2. Being Interviewed. 

Each Phase II member will be individually interviewed 
by a Group #2 member. He will experience what he will 
be asked to do with four other persons. 

Time: 2 days (2 Interviews per day) 

Stage 3. Interviewing. 

Each Phase II person will interview four assigned per- 
sons # 

Time: 2 days (4 -§days in any order) / 



Phase II Members : i 

2 District Directors 
6 Executives 
12 Professionals 

20 Total 



i 
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1725 Wait Wtlson Avenue 
Chicago, KUnot* 66^40 



it 

Addison Sr. 

•n<xi 60641 777-71 12 



Leyden Unit 

0660 Franklin Avenue 

Franklin Park* lllinoia 60131 455.3224 




II.. 1 I 

Northwest Youth Outreach 

of the YMCA of Metropolitan Chicago 

Administrative Office 

4854 West Addison Street, Chicago, Illinois 60641 
(312) 777-7112-3 



PHASE III 



Phase III will be that area which will produce the largest 
number ;of interviews. It will be performed by the persons 
involved in Phase II and supervised by the persons in Phase I. 

Time: 2 days (4 -§days in any order) 



Phase III Members : 

1 President 
8 Vice-Presidents 
7 District Directors 
19 Executives 
45 Professionals 

80 Total 
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North Unit/Rnvenswood YMCA 
\725 West Wilson Avnnue 
Chicago, Illinois ^0 561 S3dO 



Leyden Unit 

9660 Franklin Avonue 

Franklin Park, Illinois 60131 466-3224 




Northwest Youth Outreach 

of the YMCA of Metropolian Chicago 

Administrative Office 

4854 West Addison Street, Chicago, Illinois 60641 
(312) 777-7112-3 



Time Estimates 









In 


In 


In 










PHASE 


PHASE 


PHASE 


Total 








I 


II 


III 




By 


PHASE 


I 


1 day 


3 days 


1 day 


5 days 


By 


PHASE 


II 


— 


6*. hours 


2 days 


2jj days 


By 


PHASE 


III' 


--- 


--- 


•§■ day 


■§• day 



PHASE I 
PHASE II 
PHASE III 

Final Report 



April 20, 1971 
May 20, 1971 
June 10, 1971 

July 1, 1971 
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I Addison St. 

I»nc*i 60*41 777-7112 



Leyden Unit 

9660 Franklin Avenue 

Franklin Park, Illinois 60^31 456-3224 



North Unit/ Ravenswood YMCA 
1726 Watt Wilson Avenue 
Chicago, Illinois 60640 54t -&UIO 




Northwest Youth Outreach 

of the YMCA of Metropolitan Chicago 

Administrative Office 

(31^)V77-7mT S,r0e, ' ChiC8B0 ' ,,linois 60641 



Sample Distribution 



Sub-Group 1 

1 District Director 

1 Executive 

2 Professionals 



Sub-Group 2 

1 DD 

1 Ex 

2 Pro 



Sub-Group 3 

£ Ex 
2 Pro 



Sub-Group 4 

1 Ex 
3 Pro 



Sub-Group 5 

1 Ex 
3 Pro 



Total 

2 DD 
6 Ex 
12 Pro 



20 ♦ 80 « 100 



Ij n | t 



r n i Addison St. 

J^RJC |l,,nCiS 60641 777*71 12 







Leyden Unit 

9660 Franklin Avenuo 

Franklin Park, Illinois 60131 466-3224 



Interviewees 



2 Vice-Presidents 

1 DD 
4 Ex 
9 Pro 



2 VP 
1 DD 
4 Ex 
9 Pro 



2 VP 
1 DD 
4 Ex 
9 Pro 



1 VP 

2 DD 
4 Ex 
9 Pro 



1 President 

1 VP 

2 DD 

3 Ex 
9 Pro 

1 P 

6 VP 

7 DD 
19 Ex 
45 Pro 



North Unit/Ravonswood YMCA 
1726 West Wilson Avenue 
Chlctgo Mli note 60640 Sdl-KMIO 
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INTERVIEW AND DOCUMENTATION GUIDE 
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• ’ INTRODUCTION . 

• . ■ . • 

The Program Evaluation Task Force has been charged vith 
the responsibility of designing a central nervous system that 
vill help the YMCA of Metropolitan Chicago operate smoothly and 
efficiently’* 

.. Two . main reasons vhy an information system is' needed 



v 



• - ;■ i. y . 

• 






•i . 
<‘.v\ 



ares 

. (l) Community 'needs directly served by individuals 

f working "in the field” can be better met if the \ 

total resources of the "Y" can be made available, 

,* ■ •»’. 

the various' operational 'and support groups, 

V district directors , • center directors, arid field 
personnel, and 
v’Vv>. (2) Competition for public and private funds can be 
. met if "Y" personnel can construct an 

,• v/:*s>\ -v- ‘ -• 

" •* • i *' 1 * ► . ! * 

' - accurate picture of the quality and quantity of 

# * > 

• ' * • the human resources provided for the dollars 

invested. ; , '• ’’ ; 

i\ IsrsfcauM-.-tian flowing upward and downward through the. 

*• ‘ t . ; ■* 

V * ’ • . . • „ 

. system c>-*' help make thr ."Y’.'producta more useful to. the. 

*. •= «■ ;■ ..v ir>.> ... ■ ” •’ ’ I- ,■ 

-* - if. v 



1 ' 



*\\\* ' .* • 



. »• 
* ■ 



.* •« - .vi.- 

. •' ■ v 
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s , V »' 
» 



•Mi 
V*‘ *. 

■ Vl 



* i ' ’ * t* 



t f, $ 



• j 

_ i , , 



-ii- 



» i ; .. 

\ ■ - 

community they serve* These wore useful produ is. In turn, con ‘ •-*.* 

\ • • 

attract the much^needed funds for carrying on the "Y" 's work. 

Investors in the "Y" want to know, and should know, what their 

* 4 / 4 

dollars are buying. 

The Task Force is keeping uppermost in its mind that : ••••" 

* . ; . j ■ * 

any reporting that is done mu3t be useful to the person doing it. • 

* ' V- 

The Task Force will constantly be striving to see that the 
information gathered gets 'back to those who provided it in \} ’<■./ 
useful, manageable form.-.vy?:^^^ * 

I : * . . , . \. «... , >.*> ; -l *«V •**.' *’ *V f.A ; .. •••*.' * . *. * * v *. * .* * *' 
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WHY A TASK FORCE? 



The Program Evaluation Task Force was established at the •; 
direction of Mr. John Boot, President of the YMCA of Metropolitan . ^ 
Chicago, on February 23, 1971. The purpose of the Task Force is 

to conduct preliminary research on "what’s happening now" at the y. 

1 vS; 

several organizational levels within the Support and Operational . 
Groups (see Appendix I: Top-Level Organizational Chart) of the. 
Metropolitan YMCA. The findings of the Task Force will be the . y 
bricks and mortar for the foundation of an information system ' < 'n' 

•• .y . • • *.. . / •* v 'v 

which will be useful to everyone from employees operating in the .• 

• • ’■.•••• :.>•*. • ** iy> ■ ■■ - V . ••• \ .■ ••• ■ •*’. , 

. field to corporate officers. •>* ..- ; i «’ ’ t . 

^ !*;. v h • 

TASK FORCE STRUCTURE ' ' ’ 



. 4 t-. 

* v*;.' 

- * V| ; * 



The fifteen-member Task Force is comprised of three 
subgroups • Group J1 Is working to design a program and membership 
statistics record system. Group 83 is working ^o develop a 
system that will be compatible with the PPBS (Planning, Programming, 



and Budgeting System) of the Community Fund. 



\ 



Group 82 is working to find out what information is. 

• recorded at present and how it is , or might be , assembled to aid 
■ •* * ' • • 
managerial personnel in planning and managing and to help individual 



-2 



workers achieve "their every-day , operational goals • The goal of 

the Task Force is to complete research in the three areas and to, 

have a substantial start toward designing an information system • ‘ 

• ■ 

for the Metropolitan YMCA by September 1, 1971* that will meet \ 
operational needs and funding requirements* 



r 



. * \ 4 ' # ’ 



TTTFORMAT1QH GATHERING PROCESS OF GROUP ti 2 



The time requested of Task Force members was originally 
one-half day per week. However, experience has shown that the 
.planning stages alone have required more than the allotted time. 
Grcup/ 5<72 decided to use a ’’chain letter” approach to a series of 

V • 

•• interviews in order to find out what information is recorded vithin 
' our organisation and how it is recorded. This ’’chain letter” 'V;- 1 -.'; ■ 
approach will help keep demands on our time vithin reason. It. 

. will also help keep our demands on your time within reason. A 

• valuable by-product of this approach is that it will provide 

• • . 

# # #• i # % m 

• people other than ourselves a chance to get a first-hand look at ; ‘v 

' .*• * • •/ 

I * W . . # * . . 

what goes on in various areas of YMCA operations other than their < 

. * • . ■ . ( 
:. . * • • • ' . t 

/ovn« K 

\ ■ * . . • - i * 

The "chain letter" process works like this: each of the 

t ' • 

• five members of Croup M2 will interview four people* Each of those 

, I . 1 ;. ' .■ . ••••■ 

four people will interview* four other people.' In allJ one-hundred ;•. 

• • ■ •■• Z- -•• .• -•I .-' ••• • ' _ • ■ • 



•i : ■:! 



. * 



•A •< . 
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people will be interviewed. The twenty people interviewed by the 
members of Group H2. will be both interviewees and interviewers • 
You, the Chosen Twenty, are the most vital cog in the chain letter 
procesB. You must help us by interviewing the four people assigned 
to you or the goal of one-hundred interviewees will not be reached! 
And we in Group $2. will know little'' more than we do fa gw about how 
the "Y" conducts its business. ' ", • 

. ' :■ ■■ , .. \ ■ ;■ r 

GROUP HZ WILL HELP YOU 

^ — ■ ii * . ■ i 



We know it would not be fair or realistic for us to inter- 
v * ew and then ask -you to interview four other people without 
our giving you a clear understanding of what you are being asked to 



do and why. We have built into the process an orientation meeting. 

«■ ' * 4 * 

for you with us so that your questions can be. answered. You will •'*’ 

* r * 

have the benefit of going through the interview process and the ' 
orientation process before you interview anyone else » ‘ • 



We expect your orientation to take two hours. Your 
being interviewed will take one-half day. Your interviewing 5 
four people will taka two days. By your working with us and 
for us for leas than three days over a two or three-week period, * 

the process; of estohiiohing" a * data ' base can be. completed in ' 

early May.'VvV! ••• V. *V : 

w , j .i , i . ' i _•••*.** * . ,*i *. . ■. * i ,** • • • i » *• •* 



-U- 

i 

Three days of your time is all that we plan to use; we 
do not intend for demands on your time to drag on indefinitely. 

This interviewing effort is a "one-shot 1 * thing to he done 
correctly, completely, one time, and finished. 

During your orientation, the Interview Discussion Guide 
trill he explained to you so that you, in turn, will he ahle to 
get the most and the best information from those you interview. 
Your clear understanding of the "chain letter" process and the 
tools you will he using is vital because your eyes and ears will 
he the movie camera and tape recorder on the scene. Your careful 
use of the forms (The Organizational Chart, The Functional Task 
Outline, and The YMCA Document Abstract) will help us digest what 

is happening without repeatedly asking you what you saw and heard. 

% 

You will receive instructions on the use of these forms during 
your orientation. 



\ 

Y 

v 
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APPENDIX III: INTERVIEWING LOGISTICS 
• / 

? The "chain letter" interviewing process is comprised 
of three phases: 

1. Pre-testing the Interview Discussion Guide by 
Group ff2 of the Task Force* 

2. Interviewing twenty (20) people -- four by each of 
the five Group (f2 members — who, in turn,, will bo 
conduct interviews in Phase 3* 

3. Interviewing eighty (80) people — four by each of 
the twenty people interviewed in Phase 2. 

In all, one-hundred (100) interviews will be conducted. 
Having the "twenty people of Phase 2 conduct four interviews each 
will keep the time demands on the members of Group ff2 of the T 
Task Force within reason, and it will keep the tiara demands on 
the twenty people in Phase 2 within reasons 

a 

The process will also serve as a learning device for 
people to gain a better understanding of the structure and oper- 
ation of; the YMCA of Metropolitan Chicago. It will provide 
individuals with the opportunity to communicate with people other 
than those encountered in normal functional relationships. 

The following pages contain brief descriptions of the 

# 

three phases including the time involved in each and a sample 
distribution of people to be interviewed in Phases 2 .and 3. 
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PHASE ".1 

Phase 1 involves the testing and revising of the tools to he used 

i • 

during the interviews . Members of Group H2. of the Task Force 
must he familiar with the tools and he sure that they are 

N 

suitable for use by the twenty people in Phase 2 to get the 

. * • I 

desired information. 

\ 

Time required: 1 day. 



PHASE 2 

Phase 2 is the crux of tha entire process. It requires great 

r 

cure and attention by members of Group //2 because Phase 2 is 
vital to the success of Phase 3. Phase 2 involves ttvoe stages; 
Stage 1. Orientation 

. . ■ Each member of Group H2 will present the 

. • . interviewing tools, explain the interviewing 

process , and see that each of his four 
people. 'knows who they will be interviewing. 

i 

Time required: 2 hours 

I 

Stage 2 . Interviews 

Each of the four people will be interviewed 
by their representative of Group //2. Each 
of the four will experience what he, in turn, 
... . will be doing with four other people. 

i • 

Time required: 1/2 day for interviewees 

. 2 days (four 1/2 days) for 

interviewers 
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PH ASE 3 \ 

Phase 3 will produce the hulk of the Interviews . The efforts 
of the twenty people doit the interviewing will produce the 
information that is so vital to the total Task Force effort. 

Time required: 1/2 day for interviewees 

.2 days (four 1/2 days) for Interviewers 



V 



hitmv. ESTIMATE TABLE 



\ 







In 

Phase 

1 


In In 

Phase Phase 

2 3 


Total 


By Phase 1 


(5 people) 


1 Day 


3 Days 1 Day 


5 Days 


By Phase 2 


(20 people) 




6 Krs . 2 Days 


2-3 A Days 


By Phase 3 


(80 people) 


— 


1/2 Day 

« 


1/2 Day 


COMPLETION 


DATES 








• 


* 

Phase 


1 


..April 26—30, 1971 


- 


- • 


Phase 


2 







Phase 3eeeees*r eMcy 10—21 , 19T1 

!• Final Report of Group ; #2 •#•*•• • «*>une 1, 1971 
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APPENDIX IV: SAMPLE DISTRIBUTION 



- GROUP #2, MEMBER #1 

1 District Director 

1 Executive 

2 Professional 



GROUP #2 , MEMBER #2 

1 Executive 
3 Professionals 



GROUP #2, MEMBER #3 

2 Executives 

2 Professionals . , 

" ' ' • \ 

GROUP UZ 9 MEMBER #h 

V 
4 * * 

3 Executives 

1 Professional 



GROUP #2, MEMBER #5 

* 2 Executives 
2 Professionals • 



interviewees 



2 Vice-presidents 
1 District Director 
5 Executives 
8 Professionals 



2 Vice-presidents 

1 District Director 

3 Executives 

10 Professionals 



2 Vice-presidents 
2 District Directors 
5 Executives 
7 Professionals 



. 1 Vice-president 
2 District Directors 
7 Executives 
6 Professionals 



1 President 

1 Vice-president 

2 District Directors 
U Executives 

8 Professionals 



TOTALS 

1 District Director 
9 Executives 
10 professionals 
20 




1 President 



8 Vice-presidents 
3 District Directoras 
2h Executives 
39 Professionals 



80 
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APPENDIX V: INTERVIEW DISCUSSION GUIDE 



Each interview will consist of three parts: 
(l) An Organizational Chart 



(2) The Functional Task Outline 

(3) YMCA Documents Abstracts . 



Having your interviewee sketch an Organizational Chart 
of his operating unit as he perceives it will help to -get the 
juices flowing"; that is, it wiil prod him to begin thinking shout 
what he does in the performance of his Job and what goes on around 



him. 

The Functional Task Outline will provide the details 
that will fill out and support the Organizational Chart. It will . 
also help the interviewee think about the documents that he receives 
from others or submits to others in the performance of his Job. 

The YMCA Document Abstracts vill help .the Task Force 
what documents circulate within the organization and how they are 



used. 

The three parts of the interview will serve' to paint a 
fairly complete picture of "what's happening now" in the total 
YMCA. The good aspects will be highlighted, and areas that could use 
some improvement and development vill be indicated. . 











V 
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CKGANIZATIONAL CHART 

/ 



Instruction: Please sketch an organizational chart 

of your operating unit. Indicate relative positions 
of all those persons with vhom you review documents* 
to plan and develop program operations. 



* A document is a report, form or record within 
a file. . i 



at 



O 
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FUNCTIONAL TASK OUTLINE 



Tn g-h-m r > ~hi on : Please outline your functions (A, 

B, C, D, etc.) and related tasks (l, 2, 3, k,etc.). 
Use the hack of this paper if necessary. * 



A. 



B. 



1. 

2 . 

3. 

k. 



1 . 

2 . 

3. 

k. 



Instruction: Circle those tasks that are docu- 

mented (initiated or received by you) for program 
planning and development, es* » (2) C^) . • 



o 
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DOCUMENT NO. YMCA DOCUMENT ABSTRACT 

« 

A document is defined as a report, form or record vithin a file* 

Department or Cente r 

* 

Name or names of documen t * v 

ft 

Source: Prepared b y C hecked b y 

Printed by band? Yes [ ] No [ ] Typewritten? Yes [ ] No [ ] 

Calculations required? Yes [ 3 No [ ] If yes , are oaluations performed ' 

by a mechanical .device? Yes [ 3 No [ 3 

•* • 

Other documents referred t o 

‘Copies: No* of conie s D estination of each con y 



If requested, how are additional copies nrenared ? 

Is document filed vithin department or center? Yes C 3 No [ 3 

If yes, original and/or cony ? 

* * . 

If filed, in vhat sequence ? 
fiov long is document retained ? 

What is the ultimate disposition of the document ? 

Bov often is file used? Daily [ 3 Weekly [ 3 Bi-Weekly f 3 Monthly [ ] 
Seasonally [ 3 Other { 3 (Please specify ) 

Bov many documents in file ? 

Bov often is document received or prepared? Daily [ 3 Weekly [ 3 Bi-Weekly I 3 

Monthly [ 3 Seasonally [ 3 Other [ 3 (Please specify) 

• \ 

Peak ? W hen ? 

Bov many are received or prepared per month ? 

Use: ‘ 



List items on document containing special coding: (If available, attach form 
containing valid codes for the coded itemB ) 

€ 3 — 
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LIST OF INTERVIEWEES 
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APPENDIX D 



Group #2 Study Population : 

YMCA CoTmmxn ity College 

Gary Nelson, Director of Admissions 

William Jackson, Director of SET— GO 

Marc Crudo , Director of Community Development 

Maynard Moore, Director of Student Financial Aid 

Gladys Knowles , Administrative Assistant 

✓ 

John Bouseman, Vice President and Dean of Academic Affairs 
Patricia Englender, Director of Adult Clubs 

Ruth Gall i not , Assistant to the Dean of Continuing Education 
Norm Young, Supervisor of College Services 

Ravenswood YMCA Center 

Larry Alvey, Program Director 

Roger Ver Velde, Executive Director 

YOU (Youth Opportunities Unlimited) 

Johnnie Walker, Unit Director 

Near South YMCA District 
Walt Worrill, District Director 

Countryside YMCA Leadership Center 
Bobbie Turcotte, Program Director 
Gary Meier, Program Director 

YMCA of Metropolitan Chicago 



64 



Wardell Haywood, Vice President of Urban Programs 

Bruce Cole, Vice President of Financial. Development, Marketing, and Comman. 

YMCA Manpower Project 
John Bates, Director 

Northwest YMCA District 

Joseph Rokinch, District Director 

Palatine Township Community Services 
Emerson Thomas , Youth Coordinator 



O 
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65 



APPENDIX E. 



LIST OF EVALUATION DOCUMENTS 



i 
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APPENDIX E 



SAMPLE OF EVALUATIVE DOCUMENTS — Selected on the "basis that they are 
not merely nose-count, cost -accounting, bookkeeping, inventory or tally 
records, but documents pertaining to program evaluation per se: 

YMCA Community College 

Student Application for Admission 

Monetary Award Application, Illinois State Scholarship Commission 
Student Application Form for A National Defense Student Loan 
Student's Application for Loan (OE— 10T0) n 
SET-GO Annual Report (June 1, 1970 — May 31 » 1971) 
x SET-GO Talent Search Contract Amendment (OE-113^-1) 

May 1971 Report on Student Financial Aid 
x YMCA Job Description (Form JD-100) — includes job strategy 
Faculty Application Form 
Faculty Application for Contract 
Faculty Application for Leave 

Application for Educational Subsidy by the YMCA 
x Functional Service Report — 1971» Community Fund 

Ravenswood YMCA Center 

x Program Project Planning - Evaluation - Improvement for activities, 
projects and special events (FM-105 Rational Board of YMCA) 

Application for Employment (PG1-10M 7*68) 

YOU (Youth Opportunities Unlimited) 
x Program Report Sheet 

P«t atine Township Youth Committee 
Survey j?W<T/l ( 1— ^ ) and YSB Profile 

ERLC , . 

i hVjr^rw beros for further analysis 
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